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1. INTRODOCTION 



This paper is concerned with the statistical aissessment of dependency 
beyond autocorrelation in the context of nonlineair time series models. The 
central theme is that residuals from models fitted according to 

inapplicable linearity assumptions can also profitably be used for further 
analysis beyond linearity. A well known property of lineax time series 
residuals is that they axe uncorrelated; in the context of lineax models 
they should also be independent, apart from the effects of parameter 
estimation with short series. However, in the context of non lineax models 
it is not often recognised that uncorrelated residuals also hold 
information concerning higher order dependence in the data. Developments 
of such a higher order residual analysis will be explored here, in 

particular for two types of nonlinear autoregressive model which have the 
usual linear Yule-Walker autoregressive correlation structure. Some higher 
order dependency correlations will be obtained. In a later paper, the 
suggestion of reversed residuals will be made, and the analysis given in 
this paper will be extended to encompass these reversed residuals. 



2. AUTOREG3^ESSION ASD LINEAR AUTOREGRESSIVE RESIDUALS 
2.1 Autoregression for Nonlinear Models 

The standard form of autoregression needs widening for use with 
nonlinear models; we consider first the standard form which is explicitly 
autoregressive in a linear additive way and then several weaker variants. 
A stationary time series {Xtl of mean ^ is assumed. 

Under the Itnear autoregresstve model, of order p, the {X^} satisfy the 
equation 

X<^— ^ ^t— 2“^ )+ . . . +cxp( X-^—p— 4 t ' t*0 ,^l,t2,... (2.1) 

where the are independent and identically distributed and m , C 2 / . . . / Op 
are fixed parameters; these parameters are also assumed to satisfy the 
condition that the polynomial 1 -a^z- ... -OpzP has all roots outside the unit 
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circle . 



A more general definition of autoregression of order p, could be the 
Itnear conditional expectation requirement that 

- ori(Xt-i-M)+<X2(Xt-.2"^)-^- • t-0, tl,±2, . . . . (2.2) 

Note that the conditional expectation is with respect to all previous X^ . 
The definition (2.1) implies (2.2) but not vice-versa. Thus this 
definition could apply to models which are not of the linear form (2.1), 
either because the are dependent, but still with E( 6^ I X^-i . Xt- 2 ' • • • 

or because the model has some other structure altogether. For instance, 
there are the random coefficient models of Nicholls and Quinn (1982), the 
exponential distribution random coefficient models of Lawrance and Lewis 
(1981,1985), the discrete distribution random coefficient models of Jacobs 
and Lewis (1983), and the gamma-beta random coefficient models of Lewis 
(1981). 

Random coefficient autoregressive models of order p taJce the general 

form 

Xt = Xt-1+At(2)xt_2+. . .+At(P)Xt_p+Bt, t=0, tl , *2 , . . . (2,3) 

where the vector of coefficients (A^^ ^ At^ ^ ^ » ■ • . , A^^ P ) is a stationary 
vector sequence of independent random variables, independent of 

X^_l ,X^_2 ' • • • ' sometimes in addition, is independent of the random 

coefficients ^ ^ at time t. A roots condition, analogous to that 

required for (2,1) is also needed. It is easy to see that such models 
satisfy the linear conditional expectation definition of pth order 
autoregression, but are nonlinear in the sense of (2.1); see also Lawrance 
and Lewis (1985) for further discussion of the meaning of nonlinearity. A 
first order case of the type (2.3) will be used to illustrate the proposed 
method of residual analysis of autoregressive nonlinearity. 

A further and weaker definition of autoregression is the requirement 

that the autocovariances of the (Xt)' denoted by {y)c)/ just satisfy 

Yule-Walker linear difference equations of the form, 
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yr-<^iyr-1^2rr-2-»-* • r-1,2,... (2.4) 

for suitaO^le constants cti,oi2f • * • tOipt this will be referred to as 
Yule-Walker autoregression. It is true for processes which satisfy (2.1) 
amd (2.2), as may be verified in the usual manner by multiplying by X^_^ 
and tadcing expectations. The reverse is not true; (2.4) does not Imply 
(2.1) or (2.2). A case in which (2.4) holds but in which (2.1) and (2.2) 
do not, is the product autoregression model of McKenzie (1982), where, with 
p»l, E( Xt IX.^-1 '^t--2 ' * • • ) ^ fractional power of X^-i; this model will 

aJLso be used in Section 4.3 as am ilustration of the proposed residual 
methods . 

A variety of other models cam satisfy the Yule— Walker definition of 
autoregression, amd yet not satisfy the linear conditional expectation 
definition. Amongst these are first order Markov chains under certain 
conditons, and others such as the semi-Markov generated processes discussed 
in Cox and Lewis (Chapter 7, 1966) and Lewis (1980). Notice that we do not 
define nonlinear autoregressive models in a constructive way. The class is 
so wide ais to make this impossible; one such class has been studied by 
Jones (1978). Rather, we require that the autocorrelations should satisfy 
linear equations, similar in structure to those satisfied by the 
autocorrelations of linear autoregressive models. In view of this, our 
suggested analysis extends, rather than supercedes, conventional methods. 

2.2 Definition and Discussion of Linear Autoregressive Residua ls 

For the analysis of time series data involving models satisfying (2.1), 
or (2.2) or minimally (2.3), the use of linear autoregressive residuals of 
order p, defined as 

(2.5) 

is suggested. This suggestion is based on the following theorem, which is 
a generalization of a result given in Lawrance and Lewis ( 1984, Section 
7.2) for p*2 . 
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Theorem. Let the stationaxy process {Xt) satisfy the Yule-Walker type 
equations (2.4). Then the linear autoregressive residuals defined 

at (2.5) are uncorrelated (although not necessarily independent). 

Proof. The autocovariances of the residuals (2.5) are 
Cov[Rt<P),Rt+r^P)] - 

-apCov[(Xt-p->x),Rt+r^Ph (2.6) 

- Cov[(Xt-M)/Rt+r^Ph-«lCovC(Xt-ji),Rt+r+l^P^]-- • • 

-apCovC(Xt-M),Rt+rV^^^ (2.7) 

Equation (2.7) follows because the (X^) process is stationary and 
consequently the {Rt^^M process is stationary. The covariances in (2.7) 
need only be considered for positive lag since autocovariance is an even 
function of r. Then the crosscovariances on the right-hand side of (2.7) 
are all of the same type ajid given by 

Cov[(Xt-^i),Rt+r(^^] “ Cov[(xt-ji)/ ((xt+r-*A)-ai(xt+r-l->^)- • • -Op( Xt+r-p-^i ) ) 1 

" Xr~«l>'r-1~- • -“^pXr-p r-1,2,... . (2.8) 

Now by the Yule-Walker equations (2.4), the e>cpression (2.9) is zero. Thus 
using (2.8) in (2.7) 

Cov[Rt(P),Rt+r(^h * 0 r-±l,±2, (2.9) 

as was to be proven. The proof is immediate for the linear autoregressive 

model (2.1), since Rt^^^ * ^t Rt-*-r^ ^ ^ definition 

independent . 

Note that the linear autoregressive residuals {Rt^^M will generally 
still be dependent though uncorrelated in nonlinear modelling of the {X^); 
with the linear autoregressive model (2.1) the {Rt^^M not only be 

uncorrelated but also independent. It ts this difference uhich util be 
exploited to explore nonlinearity in pth order nonlinear autoregressive 
processes . The dependency attributable to parameter estimation is taken to 
be small in the large scale applications we have in mind; indeed, nonlinear 
modelling of short series of data may well be hard to justify. 

The quantities Rt^^^ are autoregressive residuals in the sense that 
R^(P) is the residual of X^ after subtracting off i'ts best lineeir 
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lecist squaxes predictor in terma of ' ^t-2 ' • • • / given by 

^(P) - (2.10) 

Thus the residuals (Rt^P^) give the basic way of taking out the linear 
correlation component in models with pth order autoregression minimally of 
the * Yule— Walker * form (2.4), 

A further point worth noting about the residuals R^(P) concerns their 
crosscovariances with the X^'s. In the proof of the theorem it is seen 

that CovCXt-4/Rt+r^^^] “ ° for r - 1,2,... . However, the other half of 

these crosscovariances is non-zero. 

Example: The ( nonGaussian) linear AR( 1 ) model 

Taking (2.1) with p-»l,and p instead of the crosscorrelation of X^-p. and 

AR( 1 ) model is given by 

Corr(X^-^,Rt«j.) - ( l-p^ )^/^ p^ for r-1,2,3,...; (2.11) 

the superscript has been dropped from Rt-r' ^ will be in similar future 
use. Note further, that in this case Rt+r independent of for 

r-1,2,... . Further use of (2.11) will be made in Sections 3 amd 5. 

For the use of {Rt^^^} data analysis, the order p of the linear 
aispect of the autoregression needs to have been chosen; any of the 

availaible standard methods may still be used. In addition, of course, the 
M, ax / 0(2 ' * • ' ' ^p need to be estimated; there are at least two convenient 
possibilities; (1) the assumption of a linear autoregressive model like 
(2.1) and the use of least squares estimation, and (2) a non-model based 
approach to estimation employing the first p Yule-Walker type equations 
(2.4). The latter is suggested here; however, Tjostheim and Paulsen (1983) 
recommend (1) when dealing with modest sized samples from linear models, on 
account of serious estimation bias with the Yule-Walker estimates. Since 
the use envisaged here is primarily with nonlinear models and substantial 
sets of data, the superiority of (1) over (2) is not established or 
crucial . 
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3. ASSESSMENT OP HIGHER ORDER DEPENDENCE 



Since the linear autoregressive * residuals {Rt^^^} of (2.5) have zero 
covaxiances when (2.4) holds, but for nonlineax processes need not be 
independent, a residual analysis of this nonlineaxity can be based on em 
SLSsessment of their higher order dependence. To consider what form this 
might take, we note that the use of {X^^} has been suggested by Granger and 
Andersen ( 1978, p. 63) for bilinear models, for which in many of the simple 
CcLses the X^ have zero autocorrelations themselves. The corresponding 
initial suggestion here for a residual analysis is, as was briefly 
illustrated in Lawrance and Lewis (1985 ), to use the residuals 
Displays can then easily be made of the associated autocorrelation 
functions, scatter plots, periodograms, cumulative periodograms , etc, using 
standaxd (second order) time series software. 

However, the autocorrelations of ((Rt^^^)^) fourth order quantities 

in the original series {X^}/ which is a double jump from the second order 
autocorrelations of the series (Xt). Such quemtities will be very 
difficult to handle theoretically with most types of nonlinear model. 
The crosscorrelation function of {Rt^^^} cind is essentially 

third order and is preferred here; for previous use see Subba Rao and Gabr 
(1984) and Lawrance and Lewis (1985). Since this crosscorrelation function 
involves the variance of ((Rt^^M^)/ it also needs some fourth order joint 
moments of (X^) up to lag p. The necessary calculations of all these 
quantities for two nonlinear models will be given in Section 4. 

The behaviour of the crosscorrelation function of (Rt^^^) snd 

((r^(P)) 2} may be judged agarnst the fact that the {Rt^^^} a^re independent 
for the linear autoregressive model of order p, and hence the 

crosscorrelation function of and {(Rt^^^)^} will be zero except at 

lag zero. For the random coefficient autoregressive processes considered 
in Section 4, the suggested crosscorrelation function will be shown to 
posess a useful rut-off property; this generalizes the NEAR( 2 ) result given 
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in Lawrance aind Lewis ( 1905 ) . 

Two rather simpler crosscorrelation quantities can be proposed from 
consideration of the zero covariajice result of ( 2 . 0 ), and the additional 
fact that when the pth order linear autoregressive model holds, and 

Rt+r^^^ (r- 1 , 2 ,...) will be independent, and not just uncorrelated. 

Working in terms of {X^} adjusted for its mean 4 , which is better 
computationally and often nicer theoretically, the following quantities may 
be considered. 



Corr[(Xt-M>)^/Rt+r^^^]' r-0, tl , ±2 , . . . (3.1) 

Corr[Xt-4r ( Rt+r^ ^ ^ 1 ' r-0±l,±2,... . (3.2) 

These autocorrelations axe not equivalent in the aspects of higher order 
dependency of the (X.^) process which they assess. To see this, note that 
the covariance corresponding to (3.1) involves only third order joint 
moments of the form E[( XfM )^( X^+r-i^M ) ] ^ i- 0 , ± 1 , . . . , tp while the 

covariance corresponding to (3.2) involves additional joint moments of the 
form ECX-t"*^ )( ^ttr-i ) ] ' i-0, ±1 , . . . , ±p . Further, the denominator of 

(3:.2), by needing var { ( / is more complicated in the higher order 
moments it involves relative to the denominator of (3.1) which needs 
var{Rt^PM* However, both correlations, and particularly (3.1), axe more 
tractable than the autocorrelations of {(Rt^^M^) or the crosscorrelations 
of and {(Rt^^h^) • 

Example (Continued from (2.11)); The ( nonGaussian ) linear AR( 1 ) model. 

As an illustration of the use of (3.1) and (3.2), for the linear AR( 1 ) 

model, (( 2 . 1 ) with p -1 and p instead of ) there are the results 

‘0 r— 1,-2, . . . 



Corr[(Xt-M).(Rt-r)^] 



skew( X ) 



var(X) 

[ var( R^ ) ]V 



( 1-p^ )p^. 



0 



(3.3) 

r* 0 , 1 , . . . 

r— 1,-2, . . . 



Corr[(Xt-4)2,Rt_r] 



Var(X) 1-p’ 



(3.4) 



zr 



r- 0 , 1 . 



Note the faster geometric decrease in (3.4), for non negative lags, 
relative to (3.3); also (3.3) contains more higher moment information. The 



7 



behaviour of these functions - zero for r<0 ajid geometric decay for r>0 - 
suggests linear models with skewed marginal distributions or nonlinear 
models. Discussion of parallel results for two types of nonlinear model 
are given in section 4. 

The use of squaring in the construction of these higher order 
dependency measures is recognized as being pragmatic and somewhat arbitary; 
it does however lead to expressions involving selected types of simpler 
higher order cross moments. The end use of the higher order dependency 
measures can either be exploratory for a given data set, to ascertain 
whether there is appreciable nonlinearity present, or constructively, to 
provide evidence for fitting suitable types of nonlinear model which can 
match the observed higher order dependency. 

Earlier discussion of higher order dependence in nonGaussian Itnear 
models is given by Rosenblatt (1980) in terms of the bispectrum; aspects of 
nonlinearity and higher order spectra are briefly considered in Rosenblatt 
(1979). 
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4. HIGHER ORDER DEPENDENCY FOR TWO TYPES OP NONLINEAR HDDEL 



In this section we obtain properties of the measures of higher order 
dependency discussed in the previous section, for two specific types of 
nonlinear models. The models considered are autoregressive in the 
Yule-Walker sense of (2.4), but not in the linear sense of (2.1). 

4.1 Random Coefficient Auto r e^ r esaive Modela 

As already remarked in Section 1, a generalization of the linear 
autoregressive model (2.1) is to let the coefficients ai,a 2 /...ap be random 
variables. One general class of such models has been discussed by Nicholls 
and Quinn (1982 ) who cite Andell (1976 ) and articles in the economic 
literature. Other classes of models with random coefficients include the 
discrete distribution models of Jacobs and Lewis (1983) and the exponential 
models of Lawrance and Lewis (1981,1985). The class of random coefficient 
autoregressive processes considered here is given by 

X<- - , (4.1) 

where , . , , is independent of Xt-i/Xt-2/ • • • ^^d forms 

a stationary vector sequence of Independent random variables where 
E(At^^M*ccj for j»l/.../P satisfy the standard polynomial equation. The 
components of the vectors are not necessarily independent. For example the 
discrete distribution models of Jacobs and Lewis ( 1983 ) can be written in 
this form, and have dependent coefficients, as do the exponential models of 
Lawrance and Lewis ( 1981, 1985 ). It is easily verified that the process 

(4.1) satisfies conditional expectation autoregression (2.2) and thus also 
the weaker Yule-Walker definition (2.4); it will also clearly be 
stationary. Note as well that the standard linear autoregressive model 

(2.1) is a special case of (4.1) in which the random coefficients are 
actually constant. 

We now give a characteristic result for this type of process when 
higher order dependency is measured by Corr[Rt^ ^ ^ ^ 

Corr[Rt(P\(Rt^-j.^^M^]/ assuming that the R^^P^’s axe uncorrelated. 
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Theorem , With the random coefficient model (4.1), 



Corr[Rt^P^/(Xt-r~^^)^] Corr[Rt^ ^ ^ ^ ( Rt-r^ ^ ^ equal to zero for 

r-1,2, 

Proof ; Using the definitions (2,5) and (4.1), 

Rt(P) - (Xt-M)-ai(Xt-i-M)-. . .-Ofp(Xt-p-^) 

- ( At^ ^ )Xt-i + ( At^ ^ ^-^2 )^t-2 + ( At^ P ^-ap )Xt-p 

+ Bf( 1^1 . . . -^p )m • (4.2) 

On multiplying (4.2) by (Rt-r^^^)^ taking expectations, thus obtaining 

Cov[R^( P ) , ( R^_^( P ) )2 ] , we have a sura of p terms given by 
P 

Z E{(At<^ >^j)Xt-j(Rt-r^P))2} (4.3) 

j-1 

and a last term involving which is clearly zero. Now is 

independent of both the X^-j and (Rt-r^^^)^' r=l,2,..., which may 

nevertheless be themselves dependent. Thus the jth term in (4.3) becomes 
E( a( 3 )-aj )E(Xt-j(Rt-r^^^ )^) * 0 r-1,2,..., 

since E( a( 3 ) )-Oj . This completes the proof which clearly includes the 
first cross correlation mentioned in the theorem. The proof highlights the 
fact that it is the independence of the vector of coefficients 

{At<^> on previous X^*s uhich creates the effect in this type 

of model » 

The results of the theorem can be used to help validate random coef- 
ficient autoregressive models; also useful in this respect are the non- 
zero higher order residual crosscorrelations, e.g. for positive r 
Corr [Rt^ P ^ ^ ( Rt+r^ ^ ^ * These have been obtained for the second order 
autoregressive exponential process studied in Lawrance and Lewis (1985); 
similar results for any first order random coefficient model of the type 
(4.1) are given in the next subsection. 

It is worth noting that with G^^-r <^e^ined as any reasonable function of 
( X^-r / , . . . ), a similar argument to that given in the proof shows that 

Corr[Rt(P),Gt_r]*0/ r=l,2,... . (4.4) 

A result of this type is not, however, sufficient to establish, for 
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instance, that and are independent; they are dependent. 

The ramdora coefficient autoregressive structure of (4.1) is not a 
necessary condition for the results of this section to hold; this may be 
illustrated by noting that CorrCR^^ P ) , ( )^ ] zero when 

Ci2( J^)=E{(Xf;i)(Xt-r”^)^} satisfy the equations 

Ci2(r)-oxCi2(r-l)4<x2Ci2(r-2)+. . .+OpCi 2 (r-p), r-1,2,... . (4.5) 

The similarity of these equations to stajidard Yule-Walker equations (2.4) 
will be appaxent . Thus any process with this property will have 
Corr[Rt^P^/(Xt-r“M)^] equal to zero for non-negative r. 

4.2 Higher Order Dependency for First Order Random Coefficient 
Autoregressive Models 

The model to be considered is the first order (p-1) case of (4.1), now 
to be denoted as 

Xt*AtXt-i+Bt, t-0 rl,±2 , (4.6) 

in which and are independent between each t, but within themselves 
may be dependent; we also write 

a-E(At), M-E(Xt)-(l-a)-lE(Bt). 

Many basic mathematical and probabilistic properties of this equation have 
been studied by Vervaat (1979). Interest here is restricted mainly to the 
residual crosscovariances of (Rt^/^t-r) [( /Rt-r ] '^bere R^ is the 

first order residual given by 

Rt-Xt-M-a( Xt-i-^ ) . (4.7) 

It has been established in Section 4.1 that Cov( 

for r*l,2,..., and so now Cov( R^^^ ^ Rt-r ) r«0,l,2,... is obtained 

explicitly. 

The calculation begins by writing the required covariance as 
Cov( Rt 2 , Rt_i- )-E { [ ( XfM )-a( Xt-i-M ) ] 2 [ ( Xt_r-4 )~a( Xt-r-l-M ) ] ) 

“C 2 i( r )-aC 2 i( r+1 )-2a(Cixi( ^ )~*Cxn( r+1 ) ) 

+a2{C2i(r-l)-aC2i(r)) (4.0) 

where, aa defined just Isefore (4.5), 

C 2 i( r )-E(( XfM )^( ) / Ciii( r )-E((Xt-M)(Xt-i-M )( ) ) • 
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The higher order triple moment Cxn(r) is easily seen to be equivalently 
given by aC 2 i(r-l), in the present instance, and hence (4.9) becomes, 

Cov(Rt2,Rt_j.)-C2i(r)-aC2i(r+l)-a2{C2i(r-l)-aC2i(r)), r-1,2, . . . (4.9) 

For r“0, with 43«E[( )^ ] » there is the seperate result 

E( Rt^ )-( l+2a3 )M3-3aC2l( 1 ) • (4.10) 

The calculation of C 2 i(r) is effected by writing the defining equation 
(4.6) in the form 

Squaxing both sides, multiplying by and taking expectations then 

gives 

C2i( r )-a2C2i( r-1 )+2( ^a2+E( At Bt )-4a )C( r-1 ) (4.11) 

where 

C( r)aCov(Xt,Xt_r (r 23 var(Xt)/ a 2 sE(At^). 

Further simplifications of (4.11) give the recursive equation 

C 2 i( r )-a 2 C 2 i( r-1 )+)caJ^“l, r-1,2,..., (4.12) 

where 

k»2cr^ (^var( At )-K:ov( At , Bt ) } » 

Equation (4.12) has explicit solution 

C2i( r )-a2^M3+k( a2^-a^ )/( a2-a ) , ( a2^a ) 

r-0,1,... . (4.13) 

-a^^3+kra^*^ , ( a2“a ) 

Going back to (4.9) for Cov( Rt^ / Rt-r ) using (4.12), we have finally 

Cov( Rt^ , Rt-r )•( a2+a^a2-aa22-a2 )C2i( r-1 ) 



+( l-a2+a^-aa2 )ka^“^, r^l,2,... . (4.14) 

Thus, (4.14) and (4.13) constitute the explicit solution for the 
crosscovaxiances of the residuals (Rt^/Rt-r)' notice that when At is the 
constant a, the case of the standaxd first order linear model, these 
crosscovariances axe correctly zero. The NEAR( 1 ) model of Lawrance and 
Lewis (1901) specializes (4.6), by having independence within each (At/Bt) 
and the particular forms 

^ r^3 w.p. a o.trTT P 

^ lo w.p. 1-a' ^ t t' t [b w.p. 1-p 



where 



(l-a)/3, p - (l-/3)/Cl-(l-a)/3) 



and (Kt) is an independently distributed exponential sequence with 
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parajneter X. 

If the covariances (4.14) aure to be converted to correlations, then 

Var( ) and Var(R-t*) i^ust be obtained. This requires the following 
calculation, 

Var(Rt^) - E(Rt*) - CE(Rt^)]* (4.15) 

where 

E(Rt*) - (1 - a^)<r2 - Var(Rt), (4.16) 

E(Rt^) - (l+a*)M4 - 4aC3i(l) + 6a2C22( 1 ) - 4a»Ci3(l), (4.i?) 

with 

ji4-E[(Xt-u)'^], Cij(l)-E[Xt-M)^(Xt_i-M)3], (i, j )-(3,l),(2,2),(l,3).(4.18) 
The joint moments in (4.17) are now obtained in terms of the first four 
moments about the mean of X^, together with the first three moments and 
joint moments about zero of and B^. Thus in terms of the further 
quantities 

a 3 - E(At’), b 2 - E(Bt*), dij - E(AiBi), ( i, j )-( 1 . 1 ), ( 1 , 2 ) , ( 2 , 1 ) (4.19) 

we have, as with the derivation of ( 4 . 11 ), that 

C 3 i(l) - 3 ^x^ar^a- 3 ^x(M 3 *»- 2 Ma^^)a 2 ‘»-( 44^3443+34^(7^ )a3 

~€4C7^dj^2'*"3o'^dj^2‘*"3( M3+24C^ )^ 21 ' (^» 20 ) 

C 22 ( 1 ) • -4^0'^-2443a+(44‘^“2443‘*'^A^<^^ 

+<T^b2+2( 43 -*-Mct^ )<3 1 1 / (4,21) 

^13(1) - ^^ 4 ^. (4.22) 

The results ( 4 . 20 )-( 4. 22 ) have been checked against an alternative method 
of calculation involving moments about zero instead of about 4 . 

It was remarked in Section 3, following (3.2), that the cross- 
correlations of (Xf 4 )^ and Rt+r^^^ also be useful in assessing higher 

order dependence. In the present case of first order residuals and 
autoregression there are the results, 

CovC(Xt-4)^. Rt-r] “ C 2 i(r) - a C 2 i(r+ 1 ), (4.23) 

Var(Rt) - (l-a^)cr2, Var[( Xf 4 )^ ) ] - M 4 ~ (4.24) 

These formula can all be applied to the NEAR( 1 ) model of unit mean by 
simply noting that in this case 
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E(At^) - a0^. E(Bt^) - rl[p+(l-p)b^]. 



(4.25) 



Figures 1 amd 2 give the results of computations of Corr[R<-* , R^-r ] and 

Corr[(Xt-M’)^/Rt-r] ^he NEAR( 1 ) model. The top left frame of Figure 1 is 

pn ■ ‘ n; . 1 

the linear EAR( 1 ) cau3e for which all cross-correlations apart from lag zero 

are ' zero; the other three cases in Figure 1 each have zero 

crosscorrelations at negative lags, in agreement with the theoretical 

results in Section 4.1, but have some non-zero values at the zero and 

positive lags. It is evident that the lag zero cross correlations contain 




a-O 910. 32-1 





Figure 1, Four computations of the crosscorrelations, for various lags, 
befsjeen the linear autoregressive residual, and (Rt^P^ for the 
NEAR( 1 ) process uith p(l)=ap held constant at 0,75; in effect the remaining 
free parameter is being varied through its allouable range. 
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much of the discriminating information between the four cases; this will be 
so for first order autoregressive models in which much of the higher order 
dependency is at lag one, and the lag zero cross correlation involves both 
third and fourth order aspects of the lag one dependency. For further 
information on the TEAR( 1 ) and PR£AR( 1 ) cases see Lawrance and Lewis 



( 1981), 

Figure 2 gives Corr[( /Rt-r 1 same four cases used in 
Figure 1. The negative lags are again zero for all cases, and hence this 
property does not discriminate the linear from the nonlinear cases. Rather 
it is the strength of the crosscorrelations at positive lags which performs 
this task, albeit less clearly than the crosscorrelations of the residuals 
and squared residuals. 



a*0 399. i-0 "55. Cose 



a-0 910, 32A 



o 

u 

yi 

rt 

O O - 

'3 



' -:o 



o 

u 

<A 



0 

loqir) 



:o 




0 

Laq(r) 

o-O '53. 3-0 990; TCiO- ' Ccs- 




rtgure 2. Four computations of the crosscorrelations , for various lags, 
between the linear autoregressive residual, f 

NEAR(i) process uith held constant at 0.75; in effect the remaining 

free parameter is being varied through its allouable range. 
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4.3 Higher Order Dependency for t±ie Product Autoregression Model 

Another form of nonlinear autoregressive models called PAR(l), was 
introduced by McKenzie ( 1982); the basic idea of its construction is to 
consider the exponentiation of the standard AR( 1 ) equation. Then its 

p 

additive structure becomes multiplicative, and the general form of the 
PAR( 1 ) product autoregressive model model is 

t-0,±l,±2 (4.26) 

where p is the dependency parameter, 0<p<l, and (B^} is an independent and 
identically distributed innovation sequence, independent of ' • • • • 

McKenzie studies the loodel when is chosen so that {X^} has a gamma 
marginal distribution, and obtains several basic results; for instance, 
that p still represents the lag one autocorrelation aind that the 
autocorrelations in general satisfy the Yule-WalXer first order equations 
((2.4) with p-1), so that p(r)*p^. It is apparent, however, that the 
linear conditional esq^ectation definition of autoregression (equation 
(2.4)) is not satisfied because of the power form implied by (4.26). 

As with the first order random coefficient raodels in Section 4.2, we 
consider the first order residual R^, given by (4.7), and will likewise 
determine Cov( ) for r=0, ±1, ±2 , , , . , noting that for this model these 

correlations are non-zero for all lags. We will use the general expression 
(4.8) in terms of the third order central moments C 2 i(r) and Cx 3 ^j^(r), but 
this time there is no simple relation between them, and both are needed for 
all lags. Also, calculation of C 2 i(r) and must be in terms of 

their uncentered components, since these axe the quantities which can 
immediately be determined from the PAR(1) equation (4.26). The required 
uncentered moments will be written as 

e2=E(Xt2), e3=E(Xt^) 



62 i( r )=E( Xt^Xt-r ) . eiii( r )-E( XfcXt-iXt-r ) 

for r*0, ti, t2, . . . , and there is need to note the special Ccises, 



(4.27) 



e2i(0)=e3/ eiii( 0 )se2i( 1 ), ein( 1 • 
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All these quantities will be calculated. 



First, it is necessary to note the following relations between the 
centred and uncentred moments, 

C 2 i( r )-e 2 i( r r (all r) 

Clll( 0 )-C2i( 1 ), Ciii( 1 )-C2i( -1 ) (4.20) 

Clll( ^ )*^lll( ^ )-M^-m(C( r )4C( r-1 )4C( 1 ) ) , ( ) . 

The calculations of e 2 i(r) amd em(r) need to be treated separately 
for positive auid negative lags, but follow in the same general manner and 
will be illustrated by that for e 2 i(r) for positive lags. By iterating the 
PAR( 1 ) equation (4.26) r steps bac)cward, 

r-1 

Xt - n 

i-0 

where p( r )*p^, r»0, 1, . . . . Squaring this equation, multiplying it by X^-r 
amd taking expectations, gives 

r-1 

e2i(r)-E(Xt-r^^(^>'*‘^) n E(Bt-i2P( ^ ) ) . (4.29) 

i-0 

To obtain the expectations in the repeated product, taking the 2p( i )th 
power of (4.26), leads to 

E{Xt^^^ i>)-E{Xt-i2^^( ^))E(Bt2P( i)), 
and hence, dropping the unnecessary suffix t, to the result 

E(b2p( i))-E(x2P( i))/E{x2P( i+D) . 

Now (4.29) can be expressed purely in moments of X, as 

e2i( r)-E{x2P( r)+l)E(x2 )/E{x2P( r)) . (4.30) 

To proceed further, invoke a gamma marginal distribution for X with density 
f(x)-0'3x^"le~®Vr(B), e,0>O, x>0; (4.31) 

this has mean d/&, variance 0/&^, third central moment 20/9^, and there is 
the )cth moment result E( X^ )-f( <3+)< )/e^r( 0 ) . From (4.30) and similar 

expressions we then have 



e2l(J^) * 1 


f0(0+l)(0+2pr)/e3. 


r-1,2, . . . 


(4.32) 


L0(0+pl^' )(0+plrl+i)/e3, 


r— 1,-2 


eiil(r) - 1 


f0( 0+p )( 0+p^-^+p^ )/e3 , 

L0( 0+p ' 1 )( 0+P+P ' 1 )/e3 , 


r-2,3, . . . 
r— 1,-2 


(4.33 ) 
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with 021 ( 0 ), eiii(O) and ein(l) being given by the special cases of 



(4.27). Use of (4.28) gives finally the required expressions 



f2/3pVO^ 

C2l(r) - [^(p|rUp2|rl)/e3, 



- [^{p|rl+l+p2|r|+l 



)/©- 



r-0,1, . . . 

r— 1,-2, . . . 

r-2 , 3 , . . . 

r— 1,-2, . . . 



(4.34) 



(4.35) 



All these may be used in (4.8) to obtain the desired result for 
Cov(R^-^,R^_j.) as 



Cov( Rt ^ , Rfc-r ) “ 



f2/3( l-p2)2pV©’ 

/3( 2-6p^+p*+3p* )/©* 

-/3( l-p)( l-p2 )2p2 I r 1-1/03 



r-1,2 

r-0 

r—1,-2, . . . 



(4.36) 



The simpler covariance of (X-^-m)^ and Rt-r also follows via (4.23) and 
(4.35) as 



f2/3( l-p2)pr/©=* 



r-0, 1, . . . 
r— 1,-2, . , . 



(4.37) 



Cov{(Xt-<x)^Rt-r) = i:0(i_;);iirT-i/e3 
Conversion of these covariances to correlations requires Var(Rt^) as at 
(4.15) and Var [ ( XfpL )^ ] as at (4.24); the required intermediate results are 

pi4 * 3/3(/3+2)/e^, 

C 3 i(l) - 3/3(0+2)p/e^, Ci 3 (l) - /3{3/34(l+p)(2tp))p/e^, 

C22(l> - {/3^+2/3p+2/3(P+2)p^}/e^. (4.38) 

These then give as at ( 4. 15 )-( 4. 18 ) the explicit expression 

Var(Rt^) • 2/3((/3+3-2(/3+6)p^-4-6p^+(/3+ll)p^-6p5-2p®}/e^. (4.39) 

Together with Var(R^) which is simply (l-p^)/3/9^, Cov( R^^ , R^«j. ) can be 
converted into Corr( Rt^/ Rt-r ) • Similarly, (4.37) requires the result 

Var{(Xt-pi)^} - 2/3(/343)/e^ (4.40) 

in order to obtain corr{(Xt-pi)^,Rt ) . 

Figures 3 and 4 illustrate the residual and squared residual cross 
correlations for the PAR( 1 ) model. Figure 3 shows how these residual cross 
correlations vary over the range of p values from p=0 to p^.9 for an 
exponential marginal distribution. At p-0, the PAR( 1 ) is an IDD process 
and the residuals are trivially independent, resulting in the only non-zero 
crosscorrelation at lag zero. As the p value increases the dependence 
spreads out, mosc strongly at positive lags. An interesting feature is the 
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lag zero crosscorrelation which changes from being strongly positive at p— O 
to approximately zero at p-0.75, to moderately negative at p-O . 9 
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rfgure 3. Computattons of the residual crosscorreiattons , Corr ( • ^) 
for the PAR( 1 ) model. Since 0^1 the marginal distribution is exponential , 
77ie lag one serial correlation is increased from p(l)^0,0 (upper left) to 
p(l)^.9 (louer right). 



Figure 4 gives four different gamma cases of the PAR( 1 ) residual cross 
correlations, all with p-O. 75. The gamma shape parameter 0 takes the 
values 0.5, 1.0, 2.5 and 10.0; these cases indicate that changes in the 
gamma parameter cause only slow changes in detail of the cross correlations 
which from the formulae ( 4.37 ), ( 4.39) and (4.40) all tend to zero as 
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/J-0.500. p-0.750. 5-1.0 




/3-1 000. p-0.750. 5-t 0 



(5-2.500. p-0.750. 5- t 0 





|5» 10.000. p-0.750. 5-1.0 




F'igure 4, Computet 'toris of the r^stdiioV crosscorreZ’Gttons , Corr( t t 

for the PAR(l) model. The p for each case of 0,75, and the figures 
illustrate the effect on the residual crosscorrelations of changing the 
index 0 of the gamma distribution through 0,5, 1,0, 2,5 and 10,0, 



5 . OONCLDSIONS 

A methodology for analyzing higher order dependence in nonlinear time 

series with pth order autoregressive correlation structure has been 

proposed. It utilizes standard uncorrelated linear autoregressive 

residuals, and the crosscorrelation function of these residuals and their 

sguares . The behaviour of this crosscorrelation function has been utilized 

for two rather different types of nonlinear model: random coefficient 

autoregression and multiplicative autoregression; the behaviour has been 
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shown to allow discrimination between models in the saune claiss with the 



same marginal and autocorrelation structures. 

The residuals crosscorrelation function provides a partial analysis of 
third order information in the time series; it does not attempt to capture 
all third order information, which is the aim of such techniques a-s 
bispectral analysis amd which will often be intractable with nonlinear 
models. Being based on standard lineax residuals, the ainalysis extends 
rather than replaces conventional residual analysis. 

Developments of the analysis which focus on the directionality implicit 
in many time series are being investigated; reversed residuals assume a 
reversed directionality and allow exploration of the consequences of such 
an asssumption. 
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•V 
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V 
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